Skip to content

feat: GSM Secrets - Keep new secrets in an optional store - #474

Open
yuechao-qin wants to merge 5 commits into
masterfrom
gsm-t1-store-hook
Open

yuechao-qin wants to merge 5 commits into
masterfrom
gsm-t1-store-hook

Conversation

@yuechao-qin

@yuechao-qin yuechao-qin commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

What

Secrets can now live outside the database: give setup_routes a secret_store, and new secrets go to it. A secret already in the secret table stays there and works as today until it is moved. Without a store, nothing changes.

Route, with a store Secret still in the table Secret not in the table
POST /api/secrets/ 409, as today create (409 if the store has it)
PUT /api/secrets/{name} the row is updated, as today update (404 if missing); an empty description keeps the stored one, as today
DELETE /api/secrets/{name} the row is deleted, as today, plus any copy in the store delete (404 if missing)
GET /api/secrets/ not listed list_secrets

SecretStore is a Protocol with create, update, get, list_secrets and delete; a store never sees the session or the table. StoredSecret carries the table's columns; its value is left out of repr, so a logged secret never shows it.

With a store, PUT and DELETE read the row with FOR UPDATE. The lock is on that one row (the lookup is by primary key), so a bulk move of the same row either finishes first or waits. DELETE also deletes the store's copy, in case a bulk move stopped after copying the row but before deleting it; otherwise a deleted secret could come back.

Why

Secret values sit in the database as plain text, so anyone who can read the table or a backup can read every secret. This lets a deployment keep them in a store of its choice, and the backend stays free of any provider.

The routes never move anything, so a request does only what it says. The next two PRs read from the store at launch and add an admin route that moves the remaining rows in bulk; until that runs, secrets still in the table are missing from the list but keep working everywhere else.

⚠️ Once rows have moved, the store is the only copy: running without the store afterwards would show those secrets as missing.

Verification

  • tests/test_secret_store.py (15 tests, fake in-memory store): no store keeps the table behaviour; create writes no row; create of a name still in the table is 409 and changes nothing; update of a row updates it in place; update of a stored secret keeps its created_at and description; an empty description keeps the stored one; update of a missing secret is 404; delete of a row, of a row with a leftover store copy, and of a stored secret, then 404; list reads only the store; the row is locked with a store and not without; the value never shows in repr; setup_routes passes the store through (200, 409, list).
  • Mutation-checked: dropping either lock, locking without a store, sending a row's update to the store, leaving the store copy on delete, a 404 when deleting a row the store does not have, a 404 after deleting a stored secret, and list ignoring the store each turn a test red.
  • Full suite: 657 passed, 22 failed — the same 22 in tests/instrumentation/test_execution_tracing.py fail on master. Also passes on Python 3.14.

@yuechao-qin
yuechao-qin requested a review from a team November 18, 2026 01:24
@yuechao-qin
yuechao-qin requested a review from Ark-kun as a code owner November 18, 2026 01:24
@yuechao-qin yuechao-qin changed the title gsm t1 store hook feat: GSM Secrets - Save secret values through an optional store hook Nov 18, 2026
@Ark-kun

Ark-kun commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

@yuechao-qin How about replacing the whole SecretsApiService, not using the Secrets DB table at all?
Have interface like SecretsStore with same methods as SecretsApiService. When passed to setup_routes or Orchestrator it basically replaces SecretsApiService.

…s on first use

SecretStore is an abstract base class whose subclass methods are wrapped to move the caller's rows out of the secret table first. Replaces the storage-type hook: no storage flag, no 503 handler.
@yuechao-qin yuechao-qin changed the title feat: GSM Secrets - Save secret values through an optional store hook feat: GSM Secrets - Keep secrets in an optional store; move table rows on first use Nov 19, 2026
Launches read the table first, then the store, without writing; get no longer takes a session.
…o the store

SecretStore is a Protocol again; no move on use. PUT/DELETE lock the row only with a store; DELETE also removes a store copy; list reads the store.
@yuechao-qin yuechao-qin changed the title feat: GSM Secrets - Keep secrets in an optional store; move table rows on first use feat: GSM Secrets - Keep new secrets in an optional store Nov 19, 2026

def create(self, *, secret: StoredSecret) -> StoredSecret:
"""Saves a new secret and returns it without its value; raises `errors.ItemAlreadyExistsError` if it exists."""
...
Keeps the stored `created_at` and name, and the stored description when `description` is None.
Raises `errors.ItemNotFoundError` when there is no such secret.
"""
...

def get(self, *, user_id: str, secret_name: str) -> str | None:
"""Returns the secret's latest value, or None when there is no such secret."""
...

def list_secrets(self, *, user_id: str) -> list[StoredSecret]:
"""Returns the user's secrets, without values."""
...

def delete(self, *, user_id: str, secret_name: str) -> None:
"""Deletes the secret and all its values; raises `errors.ItemNotFoundError` when there is none."""
...

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe, secret_interfaces.py? I already see problematic cases in the code: secret_store: secret_store.SecretStore

"""Returns the secret's latest value, or None when there is no such secret."""
...

def list_secrets(self, *, user_id: str) -> list[StoredSecret]:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naming consistency: This is the only method with _secrets suffix...
We could either call this one list or add suffixes to other ones.
I think list is fine. I've seen it used in other libraries. Yes, we'll have to be careful about using he builtin list in signatures, but only for this class. Which does not have any list. So, it should be fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants